Validation and generalizability of machine learning prediction models on attrition in longitudinal studies

Attrition in longitudinal studies is a major threat to the representativeness of data and generalizability findings. Typical approaches address systematic nonresponse are either expensive unsatisfactory (e.g., oversampling) or rely on unrealistic assumption missing at random multiple imputation). Thus, models that effectively predict who most likely drops out subsequent occasions might offer opportunity take countermeasures incentives). With current study, we introduce model validation approach examine whether attrition two nationally representative panel can be predicted accurately. We compare performance basic logistic regression with more flexible, data-driven machine learning algorithm—gradient boosting machines. Our results show almost no difference accuracies for both modeling approaches, which contradicts claims similar survey attrition. Prediction could not generalized across surveys were less accurate when tested later wave. discuss implications these findings retention, use complex algorithms, give some recommendations deal study